Repository navigation
Conversation
284km
left a comment
284km
left a comment
There was a problem hiding this comment.
Reviewed ad15079 against e45622a. No actionable findings introduced by this change.
The dedicated titlecase/swapcase exceptions and identity-preserving fold entries address the mapping differences directly. I also checked the case-method dispatch, bang-method behavior, legacy-encoding path, and every updated call to the double-byte validator.
Validation:
- Fresh O2 build with the CI-pinned Mere
991f299bcompiler and mgz revision. - Regenerated all five tables with Ruby 4.0.9: every table matches exactly, including 86 identity fold exceptions and 77 identity titlecase exceptions.
- The new corpus regression matched Ruby with indexing ON/OFF, normally and with
MERE_RUBY_GC_STRESS=100. - Exhaustive single-byte probes across the 18 affected encodings and all 256 byte values matched all 78,336 reference rows. These cover ordinary/:ascii/:turkic/:lithuanian options, folding, result bytes/encoding/validity, and bang results.
- Mixed-string probes over the 401 distinct fold/title/swap exception codepoints matched all 20,050 mapping rows across UTF-8, UTF-16LE/BE, and UTF-32LE/BE, including ASCII/Turkic modes and folding.
- The repository runner completed the focused upstream case-mapping/options files: 18/21 tests matched. The three residual failures also reproduce on the earlier baseline; the mapping file improved from 13/18 to 17/18. These residual compatibility failures are not attributed to this patch.
- CI for this head reports 523/523 corpus programs matching Ruby and a successful Linux build. The comprehensive Unicode file is assigned to shard 5, which remains pending along with other jobs.
git diff --checkpassed.
I did not repeat the full comprehensive Unicode file or Linux execution locally. The reported full-suite result and its narrow margin under the 600-second limit still need the pending CI measurement to confirm the available headroom.
|
Thanks for the detailed table regeneration and encoding checks. No code changes are needed for the two positive findings. The remaining concern is confirmed by the CI artifact: the comprehensive Unicode file reached the existing 600-second wall limit after 354 cases, so I am keeping this PR in draft. I am measuring it with the temporary-scope allocation fix in #39 before deciding the integration order. |
ad15079 to
315d208
Compare
Unicode case conversion used uppercase-plus-lowercase as a substitute for titlecase, applied final-sigma context rules that Ruby does not use, and omitted identity exceptions from case folding. Correct
capitalize,swapcaseanddowncase(:fold), including Cherokee, titlecase digraphs and Turkic cases. Preserve the receiver encoding when ASCII input maps to non-ASCII bytes. Accept the valid single byte 0x80 in GBK/CP936 and CP949 during input validation.Generate titlecase and swapcase exceptions from the pinned reference Ruby, retain identity folding exceptions, and generate a 490-byte page mask to avoid hexadecimal-key construction and table lookups where no mapping exists. The mask includes every table entry, including identity exceptions. Regressions cover UTF-8/16/32, single-byte encodings, bang methods, multibyte validation and unmapped Unicode pages.
Depends on #39; merge that PR first. The temporary-scope fix reduced the complete Unicode run from 590.6 to 539.5 seconds locally, preserving all 422 tests and 2,901,771 assertions with zero failures/errors/skips. The final head, including the page mask, also passes in 549.0 seconds with 2,005,648 KiB peak RSS. This does not establish an additional full-suite speedup from the mask; CI still needs to confirm the remaining headroom. The original head exceeded the CI wall limit, so this PR remains draft until the final-head CI confirms completion under the unchanged 600-second/2-GiB bounds.
Validation:
source = "漢字𠀀🙂A"; 100_000.times { raise unless source.swapcase == "漢字𠀀🙂a" }) reduced case-walker region allocations from 96,914,128 to 39,200,000 bytes with the page mask. One local run took 0.47 → 0.42 seconds; the allocation count is the stronger evidence, not a full-suite timing claim.